Context-Limited Reasoning

Despite the progress being made on generally-fast algorithms for solving intractable problems, the demands of the real world require the capability to produce reasoners that are guaranteed to return answers, however preliminary, in a guaranteed amount of time. We have been exploring an approach to this problem involving limiting the reasoning context to a current focus of attention. Preliminary results using this approach have been encouraging. However, a semantical understanding of such tractable reasoning mechanisms is necessary in order to understand what confidence can be placed in them: what do they and do they not tell you? We are developing a semantic framework for tractable context-limited reasoning and experimenting with algorithms that implement this semantics. In recent years, tremendous progress has been made on generally-fast algorithms for solving intractable problems (Crawford & Auton 1996; Ginsberg 1993; Selman, Levesque, & Mitchell 1992; Selman, Kautz, & Cohen 1993), and there is every reason to believe that even better performance will be obtained in the future. However, the demands of the real world require the capability to produce reasoners that are guaranteed to return answers in a guaranteed amount of time, even if those answers are preliminary or subject to error. Our basic thesis is that reasoning should be limited to a current focus of attention. Preliminary results using this approach, reported in (Etherington & Crawford 1996), have been encouraging. However, there are many important problems still to be addressed, including how a context should be developed and adapted, the tradeoffs between investing significant reasoning effort in the context and doing "lightweight" reasoning (such as unit propagation) over the whole knowledge base, and a general semantic understanding of contextlimited reasoning. We are now focusing on these issues. In particular, we have been exploring tractable reasoning mechanisms that support unit propagation as a kind of base-case and seamlessly scale up to full sound 1Copyright (~) 1997, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. 2This work was supported in part by NSF grant IRI-9412205, DARPA/Rome Labs contracts F30602-93-C-0031 & F30602-95-1-0023, and AFOSR contract F49620-96-1-0335. and complete reasoning as additional computational resources are made available. In keeping with our intention to produce a semantically well-founded reasoning architecture, we have developed a semantics that appears to characterize unit propagation, as well as supporting integration of progressively more accurate levels of reasoning. We believe that this semantics also characterizes Dalal’s (1992) notion of "k-consistency," a rich framework for limited reasoning, and has intuitive connections to other frameworks for tractable reasoning, including and Cadoli and Schaerf’s (1992). In our most recent work, we have generalized our semantics to the first-order case. Interestingly, contexts emerge as an even more critical part of the semantics at the first-order level. A context-based limitation on the domain objects over which universal quantification is scoped is critical to our guarantee of tractability. Further, the semantic valuation mechanism may help to identify those individuals that should be brought into the context in order to more accurately determine the consequences of a theory. Our semantics is 3-valued (T, F, and U or unknown), but differs from previous 3-valued semantics by introducing nondeterministic choices. Previous attempts to characterize tractable deduction using 3and 4-valued semantics (c.f., (Frisch 1987; Patel-Schneider 1985)) have had little practical effect, since they have been too restrictive, foregoing even unit propagation in their efforts to avoid the type of chaining that makes general reasoning intractable. Our framework licenses the inference of R from P, P D Q, and Q D R which is tractable, without licensing -~P from P D Q and P D -~Q, which seems to require a more complex kind of reasoning by contradiction. There are also connections between our semantics and bilattice semantics (Ginsberg 1988), but our approach imposes weaker conditions on models, resulting in a tractable theory, while traditional bilattice approaches include propositional reasoning and so are intractable. We conjecture that, provided the explicit specification of the context is finite, our approach guarantees that first-order context-limited reasoning will remain decidable (by contrast, general first-order reasoning is not decidable), and in fact produce the same complexity classes as propositional context-limited reason104 From: AAAI Technical Report WS-97-06. Compilation copyright © 1997, AAAI (www.aaai.org). All rights reserved. ing. This would ensure that the context does, in fact, limit the reasoning process, rather than the reasoning process simply ignoring or expanding the context. A surprising result is that some individuals may be neither explicitly included in nor excluded from the context (essentially it is "unknown" whether they are in the context). These individuals constitute a sort of "boundary area" which may influence the results of reasoning, but cannot force the consideration of other individuals. The existence of this boundary area is critical to tractability as it prevents recursive axioms (such as those stating that if someone is a person, her parents are also people) from generating infinite reasoning chains. It may also be useful in identifying points at which the context could be expanded to increase the accuracy of reasoning. The formal approach we are taking to first-order context-limited reasoning also connects with our earlier work on limited scope in default reasoning (Etherington, Krans, & Perlis 1991). This is surprising, since that work was motivated by quite different.concerns-avoiding certain paradoxes that result from defaults that are locally reasonable being globally inconsistent. That work also applied only to limiting the application of defaults, rather than to limiting the underlying reasoning engine. However, we believe that the results developed there on mechanisms for determining the scope of reasoning can be imported to our context-limited framework; we also believe that the current approach will provide a cleaner, integrated, framework for limited default reasoning that will capture the benefits of the earlier approach without additional machinery. Experimental results We reported on extensive experiments using randomlygenerated date in our AAAL96 paper (Etherington Crawford 1996). Our results suggest that contextlimited consistency checking is most effective for underconstrained theories (those with a clause to variable ratio of below 2, well below the so-called "critically constrained region"). Moreover, they suggest that the greatest leverage is obtained for theories where the overall density of "facts" to "rules" is relatively low. Both of these properties are conjectured to be true of commonsense theories, but we have not yet done experiments on large commonsense knowledge bases that would allow us to empirically validate these conjectures. Obtaining the large, realistic knowledge bases nec-essary for validation of our conjectures about com-monsense reasoning has proven to be problematic,apparently because of the chicken-and-egg problemthat large knowledge bases presume effective reason-ing mechanisms, but validation of such mechanismsrequires large knowledge bases. Weare still exploringpossibilities, and hope to validate our approach againststructured (non-random) knowledge bases soon.References Cadoli, M., and Schaerf, M. 1992. Approximate in-ference in default logic and circumscription. In Work-ing Notes of the ~th International Workshopon Non-monotonic Reasoning. Crawford, J., and Auton, L. 1996. Experimental re-sults on the crossover point in random 3SAT. Artifi-cial Intelligence 81. Dalal, M. 1992. Tractable deduction in knowledgerepresentation systems. In Proceedings of the ThirdInternational Conference on Principles of KnowledgeRepresentation and Reasoning (KR ’92), 393-402. Etherington, D., and Crawford, J. 1996. Towardefficient default reasoning. In Proceedings, 13th NationalConference on Artificial Intelligence (AAAI-96). Etherington, D.; Kraus, S.; and Perlis, D. 1991. Non-monotonicity and the scope of reasoning. ArtificialIntelligence 52(3):221-262. Frisch, A. 1987. Inference without chaining. In Pro-ceedings of the Tenth International Joint Conferenceon Artificial Intelligence (IJCAI-87), 515-519. Ginsberg, M. 1988. Multivalued logics: A uniformapproach to reasoning in artificial intelligence. Com-putational Intelligence 4:265-316. Ginsberg, M. 1993. Dynamic backtracking. Journalof Artificial Intelligence Research 1:25-46. Patel-Schneider, P. F. 1985. A decidable first-orderlogic for knowledge representation. In Proceedings of the Ninth International Joint Conference on ArtificialIntelligence (IJCAI-85). Selman, B.; Kautz, H. A.; and Cohen, B. 1993. Localsearch strategies for satisfiability testing. In Proceed-ings 1993 DIMACSWorkshop on Maximum Clique,GraphColoring, and Satisfiability. Selman, B.; Levesque, H.; and Mitchell, D. 1992. A new method for solving hard satisfiability problems.In Proceedings, lOth National Conference on ArtificialIntelligence (AAAI-92), 440-446.